Picture retrieving apparatus and method which converts orthogonal transform coefficients into color histogram data

ABSTRACT

The picture retrieving apparatus includes: an orthogonal conversion coefficient memory  13  of a retrieval target picture storing an orthogonal conversion coefficient of a picture targeted for a retrieval; a picture feature amount converter  2  converting an orthogonal conversion coefficient  21  read from the orthogonal conversion coefficient memory  13  of the retrieval target picture into a color histogram information; and a color histogram similarity calculator  4  calculating a similarity  42  between a color histogram information  41  of a query picture and a color histogram information  25  generated by the picture feature amount converter  2 . It unifies the kinds of the picture feature amounts to calculate the similarity between the picture feature amounts. Thus, it can specify the picture similar to the query picture from the retrieval target picture, on the basis of the calculated similarity.

TECHNICAL FIELD

The present invention relates to a picture retrieving apparatus and apicture retrieving method, and a recording medium for recording apicture retrieving program. More particularly, the present inventionrelates to a picture retrieving apparatus and a picture retrievingmethod for retrieving a similar picture after unifying kinds of picturefeature amounts in a system in which different kinds of picture featureamounts are mixed, and a recording medium for recording a program of thesame method.

BACKGROUND ART

Typically, a picture retrieving apparatus and a picture retrievingmethod using a feature of a picture have been traditionally used toretrieve a similar picture from accumulated pictures.

This kind of a conventional picture retrieving technique employs amethod of calculating a color histogram information contained in apicture and then retrieving a similar picture based on its similarity.Here, the color histogram information implies a picture feature amountcomposed of representative colors indicative of the pictures and itsconfiguration rate. As the color histogram information, it is possibleto use a histogram with regard to all colors contained in the picture, ahistogram composed of one or a plurality of representative colors andits color configuration rate, and the like.

On the contrary, an example of a picture retrieving apparatus using anorthogonal conversion coefficient of the picture as the picture featureamount is disclosed in Japanese Laid Open Patent Application (JapanesePatent Application Number; Heisei, 11-059432) filed by the sameapplicant. Here, the orthogonal conversion coefficient of the picture iscomposed of a value indicative of an energy for each band, which isobtained by dividing a frequency distribution of a spectrum of a color(average color) of the picture into a plurality of bands, and carryingout a frequency analysis by using an orthogonal matrix.

FIG. 9 is a block diagram showing the configuration of the main portionof this conventional picture retrieving apparatus, and shows the pictureretrieving apparatus including: a picture feature amount generator 102for generating a feature amount 103 from a picture data 101; a featureamount memory 104 for storing a feature amount in advance; and asimilarity calculator 106 for calculating a similarity 107 between thefeature amount 103 and a feature amount 105 contained in the featureamount memory 104.

Here, the picture feature amount generator 102 is composed of: acontraction picture generator 110 for generating a contraction picture111 from the picture data 101; a frequency analyzer 112 for performing afrequency analysis on the contraction picture 111; and a direct currentcomponent and partially alternating current component extractor 114 forextracting as the picture feature amount 103 a direct current componentand a partially alternating component from the orthogonal conversioncoefficient 113 obtained from the frequency analyzer 112.

Also, an example of the conventional picture retrieving apparatus inwhich the picture feature amounts are mixed is disclosed in JapaneseLaid Open Patent Application (JP-A-Heisei, 09-237343).

FIG. 10 is a block diagram showing the configuration of the main portionof this conventional picture retrieving apparatus. It includes: an inputunit 201 for inputting a picture feature amount; a picture featureamount memory 202 for storing the picture feature amount; and asimilarity calculator 203 for calculating a similarity between thepicture feature amount inputted from the input unit 201 and the picturefeature amount stored in the picture feature amount memory 202.

The picture feature amount memory 202 is composed of an orthogonalconversion coefficient memory 204 for storing an orthogonal conversioncoefficient of the picture; and a color histogram memory 205 for storinga color histogram information. The similarity calculator 203 is composedof a picture feature amount judging unit 206, a color histogramsimilarity calculator 207 and an orthogonal conversion coefficientsimilarity calculator 208.

The picture feature amount judging unit 206 judges whether a picturefeature amount 209 inputted from the input unit 201 is the colorhistogram information or the orthogonal conversion coefficient. If it isjudged as the color histogram information by the picture feature amountjudging unit 206, the color histogram similarity calculator 207calculates a similarity 212 between a picture feature amount 210composed of a color histogram information outputted by the picturefeature amount judging unit 206 and a color histogram information 211read from the color histogram memory 205. If it is judged as theorthogonal conversion coefficient by the picture feature amount judgingunit 206, the orthogonal conversion coefficient similarity calculator208 calculates a similarity 215 between a picture feature amount 213composed of an orthogonal conversion coefficient outputted by thepicture feature amount judging unit 206 and an orthogonal conversioncoefficient 214 read from the orthogonal conversion coefficient memory204.

However, the above-mentioned apparatuses in the respective conventionalexamples have the following various problems.

As a first problem, the conventional apparatus can not carry out acomparative retrieval between different kinds of picture featureamounts. For example, if a picture feature amount of a query picture isthe orthogonal conversion coefficient, the retrieval can not beperformed on a database including a color histogram information of aretrieval target picture as the feature amount.

As a second problem, in order to attain the retrieval corresponding tothe plurality of kinds of picture feature amounts, the similaritycalculators corresponding to the respective picture feature amounts arerequired to thereby result in a large size of the configuration of theretrieving system. Namely, if the plurality of kinds of picture featureamounts are included, it is necessary to jointly have the similaritycalculators corresponding to all the kinds of the picture featureamounts. So, the size of the similarity calculator is made larger. Also,if the plurality of kinds of similarity calculators are included, thepicture feature amounts are required for all the similarity calculators.

As a third problem, a raw picture is required when the different kindsof the picture feature amounts are generated. This is because thepicture feature amount is extracted from the raw picture.

Japanese Laid Open Patent Application (JP-A-Heisei, 10-301948) disclosesthe following method of retrieving a picture and an image. Itinvestigates a spatial distribution of a color in an exemplificationpicture proposed as a retrieval key, and thereby divides theexemplification picture into a plurality of parts (1). It divides theexemplification picture and a retrieval target picture into a pluralityof roughness as blocks, and calculates a color histogram in each block(2). It compares the part with the color histogram in the block, for theexemplification picture and the retrieval target picture, and extractsthe block constituting the part, and then generates a part pictureindicative of a region constituting the part after a hierarchicallamination (3). Next, it calculates an adjacent histogram in which aposition relation between the parts is represented as a histogram, byusing this part picture (4). Finally, it compares the adjacent histogramobtained from the exemplification picture with the adjacent histogramobtained from the retrieval target picture, and accordingly judges thepresence or absence of a subject in the retrieval target picture (5).

Japanese Laid Open Patent Application (JP-A-Heisei, 11-102439) disclosesthe following method of retrieving a subject in a picture. At first, itinvestigates a spatial distribution of a color in a retrieval keypicture in the subject, and divides into a plurality of parts (1). Itdivides the retrieval key picture and a retrieval target picture intoblocks, and calculates a color histogram in each block (2). It generatesa part picture indicating the sureness that each point in the retrievalkey picture and the retrieval target picture belongs each part, by usinga color histogram matching in which a similarity of a color is reflected(3). It calculates an adjacent histogram representative of a positionrelation between the parts, by using the part picture (4). Next, itupdates a value of the sureness with regard to the part picturegenerated from the retrieval target picture by using this adjacenthistogram, and suppresses an influence on background (5). Finally, itcompares the adjacent histograms calculated from the part pictures ofthe updated retrieval target picture and the retrieval key picture witheach other (6), and accordingly judges the presence or absence of thesubject in the retrieval target picture (7).

Japanese Laid Open Patent Application (JP-A-Heisei, 11-316819) disclosesthe following picture retrieving apparatus. With regard to each of aplurality of picture data, a picture feature amount extractor 14 dividesa picture into a plurality of blocks, and calculates a feature amountfor each block. A feature amount labeling unit 15 gives a label on thebasis of the feature amount of each block, and generates a labelhistogram information indicative of a histogram of the given label, andalso records in a picture management database 18, and then records in acomponent index 19 from which a picture IG of the picture including thelabel and the included number can be retrieved with each label as a keyA pattern matching unit 16 establishes a retrieval target label to bedefined as a retrieval condition and its range of the included number,and refers to the component index 19 under the established retrievalcondition, and then extracts the picture.

By the way, as the U.S. patents in relation to this application, thereare U.S. Pat. No. 4,646,250 February, 1987 Childress 395/149, U.S. Pat.No. 4,672,683 June, 1987 Matsueda 382,305, U.S. Pat. No. 4,716,404December, 1987 Tabata et al. 395/164, U.S. Pat. No. 4,829,453 May, 1989Katusta et al. 382/305, U.S. Pat. No. 4,944,023 July, 1990 Imao et al.382/240, U.S. Pat. No. 5,012,334 April, 1991 Etra 348/107, U.S. Pat. No.5,148,522 September, 1992 Okazaki 395/161, U.S. Pat. No. 5,179,652January, 1993 Rozmanith et al. 395/155, U.S. Pat. No. 5,202,828 April,1993 Vertelney et al. 395/159, U.S. Pat. No. 5,220,648 June, 1993 Sato395/155, U.S. Pat. No. 5,381,158 January, 1995 Takahara et al. 395/161,U.S. Pat. No. 5,421,008 May, 1995 Banning et al. 395/600, U.S. Pat. No.5,428,727 June, 1995 Kurosu et al. 395/155, U.S. Pat. No. 5,434,966July, 1995 Nakazawa at al. 395/161. U.S. Pat. No. 5,579,471 November,1996 Barber et al. 395/326, U.S. Pat. No. 5,647,058 July, 1997 Agrawalet al. 395/326, U.S. Pat. No. 5,852,823 December, 1998 De Bonet 707/6,U.S. Pat. No. 5,893,095 April, 1999 Jain et al. 707/6, and U.S. Pat. No.6,269,358 July, 2001 Hirata.

The present invention is accomplished in view of the above mentionedproblems. Therefore, a first object of the present invention is toprovide a picture retrieving apparatus and a picture retrieving methodthat can carry out a retrieval, whether a picture feature amount is anorthogonal conversion coefficient of a picture or a color histograminformation, in a system in which a picture retrieval using the colorhistogram information of the picture and a picture retrieval using theorthogonal conversion coefficient of the picture are mixed, and arecording medium for recording a retrieval program.

A second object of the present invention is to provide a pictureretrieving apparatus and a picture retrieving method that can simplifythe system configuration without any large loss of a retrievingperformance, in a system in which a picture retrieval using a colorhistogram information of a picture and a picture retrieval using anorthogonal conversion coefficient of the picture are mixed, and arecording medium for recording a retrieval program.

DISCLOSURE OF INVENTION

A picture retrieving apparatus of the present invention includes: anorthogonal conversion coefficient memory storing a plurality oforthogonal conversion coefficients of a plurality of retrieval targetpictures targeted for a retrieval; a picture feature amount converterconverting the orthogonal conversion coefficient read from theorthogonal conversion coefficient memory into a color histograminformation; and a color histogram similarity calculator calculating asimilarity between a color histogram information of a query picture andthe color histogram information generated by the picture feature amountconverter.

A picture retrieving method of the present invention includes: a step ofstoring a plurality of orthogonal conversion coefficients of a pluralityof retrieval target pictures targeted for a retrieval in an orthogonalconversion coefficient memory; a picture feature amount converting stepof converting the orthogonal conversion coefficient read from theorthogonal conversion coefficient memory into a color histograminformation; and a color histogram similarity calculating step ofcalculating a similarity between a color histogram information of aquery picture and the color histogram information generated by thepicture feature amount converting step, and wherein a similar picture tothe query picture is specified from the plurality of retrieval targetpictures, based on a plurality of the calculated similarities.

A recording medium of the present invention is a recording medium thatcan be read in by a computer, which records a program for instructing acomputer to execute the respective steps described in the pictureretrieving method of the present invention.

A picture retrieving apparatus of the present invention includes: anorthogonal conversion coefficient memory storing a plurality oforthogonal conversion coefficients of a plurality of retrieval targetpictures targeted for a retrieval; a picture feature amount converterconverting the orthogonal conversion coefficient read from theorthogonal conversion coefficient memory into a color histograminformation; a second picture feature amount converter converting anorthogonal conversion coefficient of a query picture into a colorhistogram information; and a color histogram similarity calculatorcalculating a similarity between the color histogram informationgenerated by the second picture feature amount converter and the colorhistogram information generated by the picture feature amount converter.

A picture retrieving method of the present invention includes: a step ofstoring a plurality of orthogonal conversion coefficients of a pluralityof retrieval target pictures targeted for a retrieval in an orthogonalconversion coefficient memory; a picture feature amount converting stepof converting the orthogonal conversion coefficient read from theorthogonal conversion coefficient memory into a color histograminformation; a second picture feature amount converting step ofconverting an orthogonal conversion coefficient of a query picture intoa color histogram information; and a color histogram informationsimilarity calculating step of calculating a similarity between thecolor histogram information generated by the second picture featureamount converting step and the color histogram information generated bythe picture feature amount converting step, and wherein a similarpicture to the query picture is specified from the plurality ofretrieval target pictures, based on a plurality of the calculatedsimilarities.

A recording medium of the present invention is a recording medium thatcan be read in by a computer, which records a program for instructing acomputer to execute the respective steps described in the pictureretrieving method of the present invention.

A picture retrieving apparatus of the present invention includes: acolor histogram memory storing a plurality of color histograminformation of a plurality of retrieval target pictures targeted for aretrieval; a picture feature amount converter converting an orthogonalconversion coefficient of a query picture into a color histograminformation; and a color histogram similarity calculator calculating asimilarity between the color histogram information generated by thepicture feature amount converter and the color histogram informationread from the color histogram memory.

A picture retrieving method of the present invention includes: a step ofstoring a plurality of color histogram information of a plurality ofretrieval target pictures targeted for a retrieval in a color histogrammemory; a picture feature amount converting step of converting anorthogonal conversion coefficient of a query picture into a colorhistogram information; and a color histogram information similaritycalculating step of calculating a similarity between the color histogramgenerated by the picture feature amount converting step and the colorhistogram information read from the color histogram memory, and whereina similar picture to the query picture is specified from the pluralityof retrieval target pictures, based on a plurality of the calculatedsimilarities.

A recording medium of the present invention is a recording medium thatcan be read in by a computer, which records a program for instructing acomputer to execute the respective steps described in the pictureretrieving method of the present invention.

In the picture retrieving apparatus of the present invention, thepicture feature amount converter includes: a reversely orthogonalconverter decoding a orthogonal conversion coefficient of a picture toconvert into a picture; and a color histogram information generatorextracting a color histogram information of the converted picture.

In the picture retrieving apparatus of the present invention, each ofthe picture feature amount converter and the second picture featureamount converter includes: a reversely orthogonal converter decoding aorthogonal conversion coefficient of a picture to convert into apicture; and a color histogram information generator extracting a colorhistogram information of the converted picture.

In the picture retrieving apparatus of the present invention, thepicture feature amount converter includes: a reversely orthogonalconverter decoding a orthogonal conversion coefficient of a picture toconvert into a picture; and a color histogram information generatorextracting a color histogram information of the converted picture.

In the picture retrieving method of the present invention, the picturefeature amount converting step includes: a reversely orthogonallyconverting step of decoding a orthogonal conversion coefficient of apicture to convert into a picture; and a color histogram informationgenerating step of extracting a color histogram information of theconverted picture.

In the picture retrieving method of the present invention, each of thepicture feature amount converting step and the second picture featureamount converting step includes: a reversely orthogonally convertingstep of decoding a orthogonal conversion coefficient of a picture toconvert a picture; and a color histogram information generating step ofextracting a color histogram information of the converted picture.

In the picture retrieving method of the present invention, the picturefeature amount converting step includes: a reversely orthogonallyconverting step of decoding a orthogonal conversion coefficient of apicture to convert into the picture; and a color histogram informationgenerating step of extracting a color histogram information of theconverted picture.

A recording medium of the present invention is a recording medium thatcan be read in by a computer, which records a program for instructing acomputer to execute the respective steps described in the pictureretrieving method of the present invention.

A recording medium of the present invention is a recording medium thatcan be read in by a computer, which records a program for instructing acomputer to execute the respective steps described in the pictureretrieving method of the present invention.

A recording medium of the present invention is a recording medium thatcan be read in by a computer, which records a program for instructing acomputer to execute the respective steps described in the pictureretrieving method of the present invention.

A picture retrieving apparatus of the present invention includes: acolor histogram memory storing a plurality of color histograminformation of a plurality of retrieval target pictures targeted for aretrieval; a picture feature amount converter converting the colorhistogram information read from the color histogram memory into anorthogonal conversion coefficient; and an orthogonal conversioncoefficient similarity calculator calculating a similarity between anorthogonal conversion coefficient of a query picture and the orthogonalconversion coefficient generated by the picture feature amountconverter.

A picture retrieving method of the present invention includes: a step ofstoring a plurality of color histogram information of a plurality ofretrieval target pictures targeted for a retrieval in a color histogrammemory; a picture feature amount converting step of converting the colorhistogram information read from the color histogram memory into anorthogonal conversion coefficient; and an orthogonal conversioncoefficient similarity calculating step of calculating a similaritybetween an orthogonal conversion coefficient of a query picture and theorthogonal conversion coefficient generated by the picture featureamount converting step, and wherein a similar picture to the querypicture is specified from the plurality of retrieval target pictures,based on a plurality of the calculated similarities.

A recording medium of the present invention is a recording medium thatcan be read in by a computer, which records a program for instructing acomputer to execute the respective steps described in the pictureretrieving method of the present invention.

A picture retrieving apparatus of the present invention includes a colorhistogram memory storing a plurality of color histogram information of aplurality of retrieval target pictures targeted for a retrieval; apicture feature amount converter converting the color histograminformation read from the color histogram memory into an orthogonalconversion coefficient; a second picture feature amount converterconverting a color histogram information of a query picture into anorthogonal conversion coefficient; and an orthogonal conversioncoefficient similarity calculator calculating a similarity between theorthogonal conversion coefficient generated by the second picturefeature amount converter and the orthogonal conversion coefficientgenerated by the picture feature amount converter.

A picture retrieving method of the present invention includes: a step ofstoring a plurality of color histogram information of a plurality ofretrieval target pictures targeted for a retrieval in a color histogrammemory; a picture feature amount converting step of converting the colorhistogram information read from the color histogram memory into anorthogonal conversion coefficient; a second picture feature amountconverting step of converting a color histogram information of a querypicture into an orthogonal conversion coefficient; and an orthogonalconversion coefficient similarity calculating step of calculating asimilarity between the orthogonal conversion coefficient generated bythe second picture feature amount converting step and the orthogonalconversion coefficient generated by the picture feature amountconverting step, and wherein a similar picture to the query picture isspecified from the plurality of retrieval target pictures, based on aplurality of the calculated similarities.

A recording medium of the present invention is a recording medium thatcan be read in by a computer, which records a program for instructing acomputer to execute the respective steps described in the pictureretrieving method of the present invention.

A picture retrieving apparatus of the present invention includes: anorthogonal conversion coefficient memory storing a plurality oforthogonal conversion coefficients of a plurality of retrieval targetpictures targeted for a retrieval; a picture feature amount converterconverting a color histogram information of a query picture into anorthogonal conversion coefficient; and an orthogonal conversioncoefficient similarity calculator calculating a similarity between theorthogonal conversion coefficient generated by the picture featureamount converter and the orthogonal conversion coefficient read from theorthogonal conversion coefficient memory.

A picture retrieving method of the present invention includes: a step ofstoring a plurality of orthogonal conversion coefficients of a pluralityof retrieval target pictures targeted for a retrieval in an orthogonalconversion coefficient memory; a picture feature amount converting stepof converting a color histogram information of a query picture into anorthogonal conversion coefficient; and an orthogonal conversioncoefficient similarity calculating step of calculating a similaritybetween the orthogonal conversion coefficient generated by the picturefeature amount converting step and the orthogonal conversion coefficientread from the orthogonal conversion coefficient memory, and wherein asimilar picture to the query picture is specified from the plurality ofretrieval target pictures, based on a plurality of the calculatedsimilarities.

A recording medium of the present invention is a recording medium thatcan be read in by a computer, which records a program for instructing acomputer to execute the respective steps described in the pictureretrieving method of the present invention.

In the picture retrieving apparatus of the present invention, thepicture feature amount converter includes: an average color calculatorcalculating an average color from a color histogram information, and anorthogonal conversion coefficient generator generating an orthogonalconversion coefficient with the calculated average color as a directcurrent component.

In the picture retrieving apparatus of the present invention, each ofthe picture feature amount converter and the second picture featureamount converter includes: an average color calculator calculating anaverage color from a color histogram information, and an orthogonalconversion coefficient generator generating an orthogonal conversioncoefficient with the calculated average color as a direct currentcomponent.

In the picture retrieving apparatus of the present invention, thepicture feature amount converter includes: an average color calculatorcalculating an average color from a color histogram information, and anorthogonal conversion coefficient generator generating an orthogonalconversion coefficient with the calculated average color as a directcurrent component.

In the picture retrieving method of the present invention, the picturefeature amount converting step includes: an average color calculatingstep of calculating an average color from a color histogram information;and an orthogonal conversion coefficient generating step of generatingan orthogonal conversion coefficient with the calculated average coloras a direct current component.

In the picture retrieving method of the present invention, each of thepicture feature amount converting step and the second picture featureamount converting step includes: an average color calculating step ofcalculating an average color from a color histogram information; and anorthogonal conversion coefficient generating step of generating anorthogonal conversion coefficient with the calculated average color as adirect current component.

In the picture retrieving method of the present invention, the picturefeature amount converting step includes: an average color calculatingstep of calculating an average color from a color histogram information;and an orthogonal conversion coefficient generating step of generatingan orthogonal conversion coefficient with the calculated average coloras a direct current component.

A recording medium of the present invention is a recording medium thatcan be read in by a computer, which records a program for instructing acomputer to execute the respective steps described in the pictureretrieving method of the present invention.

A recording medium of the present invention is a recording medium thatcan be read in by a computer, which records a program for instructing acomputer to execute the respective steps described in the pictureretrieving method of the present invention.

A recording medium of the present invention is a recording medium thatcan be read in by a computer, which records a program for instructing acomputer to execute the respective steps described in the pictureretrieving method of the present invention.

In the configuration of the present invention, by using the reverselyorthogonal conversion, the orthogonal conversion coefficient of theinput picture is decoded to convert into the picture, and the colorhistogram information is extracted from the decoded picture, or theaverage color is calculated from the input color histogram information,and the orthogonal conversion coefficient is generated with thecalculated average color as the direct current component, by using theorthogonal conversion. Thus, in the system in which the color histograminformation and the orthogonal conversion coefficient of the picture aremixed, as the picture feature amount, it is possible to convert thekinds of the picture feature amounts into each other. Accordingly, it ispossible to unify the kinds of the picture feature amounts. Hence, inthe system in which the picture retrieval using the color histograminformation and the picture retrieval using the orthogonal conversioncoefficient of the picture are mixed, the similar picture can beretrieved independently of the kind of the picture feature amount, andthe similarity between the pictures can be calculated only by using onekind of a similarity calculator.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram showing a configuration of a first embodimentof a picture retrieving apparatus of the present invention;

FIG. 2 is a block diagram showing a configuration of a second embodimentof a picture retrieving apparatus of the present invention;

FIG. 3 is a block diagram showing a configuration of a third embodimentof a picture retrieving apparatus of the present invention;

FIG. 4 is a block diagram showing a configuration of a fourth embodimentof a picture retrieving apparatus of the present invention;

FIG. 5 is a block diagram showing a configuration of a fifth embodimentof a picture retrieving apparatus of the present invention;

FIG. 6 is a block diagram showing a configuration of a sixth embodimentof a picture retrieving apparatus of the present invention;

FIG. 7 is a view showing a normalizing average retrieval order when anorthogonal conversion coefficient is used as a picture feature amount,in an example of the present invention;

FIG. 8 is a view showing a normalizing average retrieval order when acolor histogram information is used as a picture feature amount, in anexample of the present invention;

FIG. 9 is a block diagram (1) showing a configuration of a main portionof a conventional picture retrieving apparatus; and

FIG. 10 is a block diagram (2) showing a configuration of a main portionof a conventional picture retrieving apparatus.

BEST MODE FOR CARRYING OUT THE INVENTION

An embodiment of the present invention will be described below withreference to the attached drawings.

FIG. 1 is a block diagram showing a configuration of a first embodimentof a picture retrieving apparatus of the present invention. FIG. 2 is ablock diagram showing a configuration of a second embodiment of apicture retrieving apparatus of the present invention. FIG. 3 is a blockdiagram showing a configuration of a third embodiment of a pictureretrieving apparatus of the present invention. FIG. 4 is a block diagramshowing a configuration of a fourth embodiment of a picture retrievingapparatus of the present invention. FIG. 5 is a block diagram showing aconfiguration of a fifth embodiment of a picture retrieving apparatus ofthe present invention. FIG. 6 is a block diagram showing a configurationof a sixth embodiment of a picture retrieving apparatus of the presentinvention. FIG. 7 is a view showing a normalizing average retrievalorder to the same picture database and query in a method of using acolor histogram information extracted from a raw picture as a picturefeature amount and a method of using a color histogram informationgenerated by converting an orthogonal conversion coefficient as apicture feature amount. And, FIG. 8 is a view showing a normalizingaverage retrieval order to the same picture database and query in amethod of using an orthogonal conversion coefficient extracted from araw picture as a picture feature amount and a method of using anorthogonal conversion coefficient generated by converting a colorhistogram information as a picture feature amount.

FIRST EMBODIMENT

The picture retrieving apparatus of the first embodiment of the presentinvention includes: an orthogonal conversion coefficient memory 13 of aretrieval target picture, a picture feature amount converter 2 and acolor histogram similarity calculator 4, as shown in FIG. 1. Moreover,the picture feature amount converter 2 is composed of a reverselyorthogonal converter 22 and a color histogram information generator 24.

The orthogonal conversion coefficient memory 13 of the retrieval targetpicture stores an orthogonal conversion coefficient of a picture that isa picture feature amount of the retrieval target picture.

In the picture feature amount converter 2, the reversely orthogonalconverter 22 performs the reversely orthogonal conversion on anorthogonal conversion coefficient 21 read from the orthogonal conversioncoefficient memory 13 to output a decode picture 23. The color histograminformation generator 24 extracts a color histogram information 25 fromthe input decode picture 23.

The color histogram similarity calculator 4 calculates a similarity 42between a color histogram information 41 of a query picture inputted asa picture feature amount used for a query and the color histograminformation 25 generated by the picture feature amount converter 2.

Thus, by calculating the similarity 42 between the color histograminformation obtained by sequentially reading orthogonal conversioncoefficients of a plurality of retrieval target pictures stored in theorthogonal conversion coefficient memory 13 and performing theconversion on them, and the color histogram information of the querypicture, and then selecting the retrieval target picture on the basis ofthe degree of the calculated similarity, it is possible to specify thepicture similar to the query picture from the plurality of retrievaltarget pictures whose orthogonal conversion coefficients are stored inthe orthogonal conversion coefficient memory 13.

As mentioned above, according to the picture retrieving apparatus of thefirst embodiment of the present invention shown in FIG. 1, it calculatesthe similarity between the color histogram information into which theorthogonal conversion coefficient of the picture, which is the picturefeature amount, is converted and the input color histogram information.Thus, both the color histogram information and the orthogonal conversioncoefficient of the picture can be treated by the same similaritycalculator. Hence, it is possible to drop the configuration size of theretrieving system.

Moreover, it is possible to provide the picture retrieving apparatusthat can carry out the retrieval, whether the kind of the picturefeature amount is the orthogonal conversion coefficient or the colorhistogram information.

SECOND EMBODIMENT

The picture retrieving apparatus of the second embodiment of the presentinvention includes: the orthogonal conversion coefficient memory 13 ofthe retrieval target picture, the picture feature amount converter 2, asecond picture feature amount converter 5 and the color histogramsimilarity calculator 4, as shown in FIG. 2. Moreover, the picturefeature amount converter 2 is composed of the reversely orthogonalconverter 22 and the color histogram information generator 24. Also, thesecond picture feature amount converter 5 is composed of a reverselyorthogonal converter 52 and a color histogram information generator 54.

The orthogonal conversion coefficient memory 13 of the retrieval targetpicture stores the orthogonal conversion coefficient of the picture thatis the picture feature amount of the retrieval target picture.

In the picture feature amount converter 2, the reversely orthogonalconverter 22 performs the reversely orthogonal conversion on theorthogonal conversion coefficient 21 read from the orthogonal conversioncoefficient memory 13 of the retrieval target picture to output thedecode picture 23. The color histogram information generator 24 extractsthe color histogram information 25 from the input decode picture 23.

In the second picture feature amount converter 5, the reverselyorthogonal converter 52 performs the reversely orthogonal conversion onthe orthogonal conversion coefficient 51 of the query picture inputtedas the picture feature amount used for the query, and outputs the decodepicture 53. The color histogram information generator 54 extracts acolor histogram information 55 from the input decode picture 53.

The color histogram information similarity calculator 4 calculates thesimilarity 42 between the color histogram information 25 generated bythe picture feature amount converter 2 and the color histograminformation 55 generated by the second picture feature amount converter5.

Thus, by calculating the similarity 42 between the color histograminformation obtained by sequentially reading the orthogonal conversioncoefficients of the plurality of retrieval target pictures stored in theorthogonal conversion coefficient memory 13, and performing theconversion on them, and the color histogram information of the querypicture, and then selecting the retrieval target picture on the basis ofthe degree of the calculated similarity, it is possible to specify thepicture similar to the query picture from the plurality of retrievaltarget pictures whose orthogonal conversion coefficients are stored inthe orthogonal conversion coefficient memory 13.

As mentioned above, according to the picture retrieving apparatus of thesecond embodiment of the present invention shown in FIG. 2, itcalculates the similarity between the color histogram information intowhich the orthogonal conversion coefficient of the picture, which is thepicture feature amount, is converted, and the color histograminformation into which the input orthogonal conversion coefficient isconverted. Thus, without any inclusion of the orthogonal conversioncoefficient similarity calculator, the color histogram informationsimilarity calculator 4 can carry out the comparison between theorthogonal conversion coefficients. Hence, it is possible to drop theconfiguration size of the retrieving system.

Moreover, it is possible to provide the picture retrieving apparatusthat can carry out the retrieval, whether the kind of the picturefeature amount is the orthogonal conversion coefficient or the colorhistogram information.

THIRD EMBODIMENT

The picture retrieving apparatus of the third embodiment of the presentinvention includes a color histogram memory 14 of the retrieval targetpicture, the picture feature amount converter 5 and the color histogramsimilarity calculator 4, as shown in FIG. 3. Moreover, the picturefeature amount converter 5 is composed of the reversely orthogonalconverter 52 and the color histogram information generator 54.

The color histogram memory 14 of the retrieval target picture stores thecolor histogram information that is the picture feature amount of theretrieval target picture.

In the picture feature amount converter 5, the reversely orthogonalconverter 52 performs the reversely orthogonal conversion on theorthogonal conversion coefficient 51 of the query picture inputted asthe picture feature amount used for the query to output the decodepicture 53. The color histogram information generator 54 extracts thecolor histogram information 55 from the input decode picture 53.

The color histogram information similarity calculator 4 calculates thesimilarity 42 between the color histogram information 55 generated bythe picture feature amount converter 5 and a color histogram information31 read from the color histogram memory 14 of the retrieval targetpicture.

Thus, by calculating the similarity 42 between the color histograminformation obtained by sequentially reading the color histograminformation of the plurality of retrieval target pictures stored in thecolor histogram memory 14, and the color histogram information intowhich the orthogonal conversion coefficient of the query picture isconverted, and then selecting the retrieval target picture on the basisof the degree of the calculated similarity, it is possible to specifythe picture similar to the query picture from the plurality of retrievaltarget pictures whose color histogram information are stored in thecolor histogram memory 14.

As mentioned above, according to the picture retrieving apparatus of thethird embodiment of the present invention shown in FIG. 3, it calculatesthe similarity between the color histogram information into which theorthogonal conversion coefficient of the input picture is converted andthe color histogram information which is the picture feature amount.Thus, both the color histogram information and the orthogonal conversioncoefficient can be treated by the same similarity calculator. Hence, itis possible to drop the configuration size of the retrieving system.

Moreover, it is possible to provide the picture retrieving apparatusthat can carry out the retrieval, whether the kind of the picturefeature amount is the orthogonal conversion coefficient or the colorhistogram information.

FOURTH EMBODIMENT

The picture retrieving apparatus of the fourth embodiment of the presentinvention includes: the color histogram memory 14 of the retrievaltarget picture, the picture feature amount converter 3 and an orthogonalconversion coefficient similarity calculator 6, as shown in FIG. 4.Moreover, the picture feature amount converter 3 is composed of anaverage color calculator 32 and an orthogonal conversion coefficientgenerator 34.

The color histogram memory 14 of the retrieval target picture stores thecolor histogram information that is the picture feature amount of theretrieval target picture.

In the picture feature amount converter 3, the average color calculator32 calculates an average color of the picture from the color histograminformation 31 of the picture, which is the picture feature amount readfrom the color histogram memory 14 of the retrieval target picture tooutput an average color 33 of the picture. Here, the average colorimplies an average value of the colors included in the color histogram,and it can be calculated by multiplying each color by each rate and thenadding all of them. Also, in calculating the average color, the averagecolor may be determined after each color is converted into a differentcolor space. The orthogonal conversion coefficient generator 34 extractsan orthogonal conversion coefficient 35 in which the average color 33 ofthe input picture is used as a direct current component of theorthogonal conversion coefficient.

The orthogonal conversion coefficient similarity calculator 6 calculatesa similarity 62 between an orthogonal conversion coefficient 71 of thequery picture inputted as the picture feature amount used for the queryand the orthogonal conversion coefficient 35 generated by the picturefeature amount converter 3.

Thus, by calculating the similarity 62 between the orthogonal conversioncoefficient obtained by sequentially reading the color histograminformation of the plurality of retrieval target pictures stored in thecolor histogram memory 14 and performing the conversion on them, and theorthogonal conversion coefficient of the query picture, and thenselecting the retrieval target picture on the basis of the degree of thecalculated similarity, it is possible to specify the picture similar tothe query picture from the plurality of retrieval target pictures whosecolor histogram information are stored in the color histogram memory 14.

As mentioned above, according to the picture retrieving apparatus of thefourth embodiment of the present invention shown in FIG. 4, itcalculates the similarity between the orthogonal conversion coefficientinputted as the picture feature amount used for the query and theorthogonal conversion coefficient into which the color histograminformation of the picture, which is the picture feature amount, isconverted. Thus, both the color histogram information and the orthogonalconversion coefficient can be treated by the same similarity calculator.Hence, it is possible to drop the configuration size of the retrievingsystem.

Moreover, it is possible to provide the picture retrieving apparatusthat can carry out the retrieval, whether the kind of the picturefeature amount is the orthogonal conversion coefficient or the colorhistogram information.

FIFTH EMBODIMENT

The picture retrieving apparatus of the fifth embodiment of the presentinvention includes: the color histogram memory 14 of the retrievaltarget picture, the picture feature amount converter 3, a second picturefeature amount converter 8 and the orthogonal conversion coefficientsimilarity calculator 6, as shown in FIG. 5. Moreover, the picturefeature amount converter 3 is composed of the average color calculator32 and the orthogonal conversion coefficient generator 34. Also, thesecond picture feature amount converter 8 is composed of an averagecolor calculator 82 and an orthogonal conversion coefficient generator84.

The color histogram memory 14 of the retrieval target picture stores thecolor histogram information that is the picture feature amount of theretrieval target picture.

In the picture feature amount converter 3, the average color calculator32 calculates the average color of the picture from the color histograminformation 31 of the picture, which is the picture feature amount readfrom the color histogram memory 14 of the retrieval target picture tooutput the average color 33 of the picture. The orthogonal conversioncoefficient generator 34 extracts the orthogonal conversion coefficient35 in which the average color 33 of the input picture is used as thedirect current component of the orthogonal conversion coefficient.

In the second picture feature amount converter 8, the average colorcalculator 82 calculates the average color of the picture from the colorhistogram information 81 of the picture, which is the picture featureamount of the query picture to output the average color 83 of thepicture. The orthogonal conversion coefficient generator 84 extracts anorthogonal conversion coefficient 85 in which the average color 83 ofthe input picture is used as the direct current component of theorthogonal conversion coefficient.

Here, the average color implies the average value of the colors includedin the color histogram, and it can be calculated by multiplying eachcolor by each rate and then adding all of them. Also, in calculating theaverage color, the average color may be determined after each color isconverted into a different color space.

The orthogonal conversion coefficient similarity calculator 6 calculatesthe similarity 62 between the orthogonal conversion coefficient 35generated by the picture feature amount converter 3 and the orthogonalconversion coefficient 85 generated by the second picture feature amountconverter 8.

Thus, by calculating the similarity 62 between the orthogonal conversioncoefficient obtained sequentially reading the color histograminformation of the plurality of retrieval target pictures stored in thecolor histogram memory 14 and performing the conversion on them, and theorthogonal conversion coefficient into which the color histograminformation of the query picture is converted, and then selecting theretrieval target picture on the basis of the degree of the calculatedsimilarity, it is possible to specify the picture similar to the querypicture from the plurality of retrieval target pictures whose colorhistogram information are stored in the color histogram memory 14.

As mentioned above, according to the picture retrieving apparatus of thefifth embodiment of the present invention shown in FIG. 5, it calculatesthe similarity between the orthogonal conversion coefficient into whichthe color histogram information of the picture inputted as the picturefeature amount used for the query is converted, and the orthogonalconversion coefficient into which the color histogram information of thepicture, which is the picture feature amount, is converted. Thus,without any inclusion of the color histogram information similaritycalculator, the comparison between the color histogram information canbe treated by the orthogonal conversion coefficient similaritycalculator 6. Hence, it is possible to drop the configuration size ofthe retrieving system.

Moreover, it is possible to provide the picture retrieving apparatusthat can carry out the retrieval, whether the kind of the picturefeature amount is the orthogonal conversion coefficient or the colorhistogram information.

SIXTH EMBODIMENT

The picture retrieving apparatus of the sixth embodiment of the presentinvention includes: the orthogonal conversion coefficient memory 13 ofthe retrieval target picture, the picture feature amount converter 8 andthe orthogonal conversion coefficient similarity calculator 6, as shownin FIG. 6. Moreover, the picture feature amount converter 8 is composedof the average color calculator 82 and the orthogonal conversioncoefficient generator 84.

The orthogonal conversion coefficient memory 13 of the retrieval targetpicture stores the orthogonal conversion coefficient of the picture thatis the picture feature amount of the retrieval target picture.

In the picture feature amount converter 8, the average color calculator82 calculates the average color of the picture from the color histograminformation 81 of the picture, which is the picture feature amount ofthe query picture to output the average color 83 of the picture. Here,the average color implies the average value of the colors included inthe color histogram, and it can be calculated by multiplying each colorby each rate and then adding all of them. Also, in calculating theaverage color, the average color may be determined after each color isconverted into a different color space. The orthogonal conversioncoefficient generator 84 extracts the orthogonal conversion coefficient85 in which the average color 83 of the input picture is used as thedirect current component of the orthogonal conversion coefficient.

The orthogonal conversion coefficient similarity calculator 6 calculatesthe similarity 62 between the orthogonal conversion coefficient 85generated by the picture feature amount converter 8 and an orthogonalconversion coefficient 91 read from the orthogonal conversioncoefficient memory 13 of the retrieval target picture.

Thus, by calculating the similarity 62 between the orthogonal conversioncoefficient obtained sequentially reading the orthogonal conversioncoefficients of the plurality of retrieval target pictures stored in theorthogonal conversion coefficient memory 13, and the orthogonalconversion coefficient into which the color histogram information of thequery picture is converted, and then selecting the retrieval targetpicture on the basis of the degree of the calculated similarity, it ispossible to specify the picture similar to the query picture from theplurality of retrieval target pictures whose orthogonal conversioncoefficients are stored in the orthogonal conversion coefficient memory13.

As mentioned above, according to the picture retrieving apparatus of thesixth embodiment of the present invention shown in FIG. 6, it calculatesthe similarity between the orthogonal conversion coefficient into whichthe color histogram information of the picture inputted as the picturefeature amount used for the query is converted, and the orthogonalconversion coefficient of the picture that is the picture featureamount. Thus, both the color histogram information and the orthogonalconversion coefficient can be treated by the same similarity calculator6. Hence, it is possible to drop the configuration size of theretrieving system.

Moreover, it is possible to provide the picture retrieving apparatusthat can carry out the retrieval, whether the kind of the picturefeature amount is the orthogonal conversion coefficient or the colorhistogram information.

EMBODIMENT

The embodiment of the present invention will be described below.

In the embodiment, a database including 5466 sheets of picture is usedto carry out a retrieval experiment. At this time, it is assumed thatwith regard to 50 query pictures, similar pictures are selected inadvance by respective subjective evaluations and defined as correctanswers.

The retrieval performance is evaluated by carrying out a re-alignment inan order starting from the similar similarity between the picturefeature amounts of the picture included in the database and the querypicture, and then evaluating the number and the order of the correctpicture located within an n-th rank from a top rank as the normalizingaverage retrieval order.

By the way, the normalizing average retrieval order implies a value inwhich an average value of expected values for the orders of the correctpictures detected within the n-th rank from the top rank is normalizedbetween 0 and 1. The smaller that value, the higher the performance. Itwas assumed that the n was the numeral equal to four times the number ofthe previously selected correct pictures, and that the correct answerswhich were not detected within the n-th from the top rank were alldetected as an (n+1) rank.

The following two kinds were prepared as the picture feature amount.

The orthogonal conversion coefficient of the picture was used as a firstpicture feature amount. A generating method was carried out as follows.A static picture was divided into 8×8 blocks. An average color of eachblock was calculated to then generate a contraction picture of a fixedsize (8 pixels×8 pixels). Next, a discrete cosine conversion (DCTconversion) was performed on the contraction picture. A low ordercoefficient row of a color difference signal and a brightness signal wasextracted from the obtained coefficient. Moreover, the obtained DCTcoefficient was converted by using a conversion table, and it wasdefined as a picture feature amount 1.

Moreover, the color histogram information was used as the other picturefeature amount. One or more representative colors and their frequencieswere extracted from each picture, and defined as a picture featureamount 2.

Next, the color histogram information was generated from the generatedpicture feature amount 1 (DCT coefficient) by using the method describedbelow. The reversely orthogonal conversion was performed on the DCTcoefficient so that the picture of 8×8 pixels was generated. Then, thecolor, histogram information was extracted from the picture of 8×8pixels. The extracted color histogram information was referred to as apicture feature amount 3.

Also, the DCT coefficient was generated from the generated picturefeature amount 2 (color histogram information) by using the methoddescribed below. A product of a rate of a color frequency and a pixelvalue was determined for each representative color, and its sum was usedas an average color information, and defined as a direct currentcomponent of the DCT coefficient. Accordingly, the DCT coefficient wasgenerated. The generated DCT coefficient was referred to as a picturefeature amount 4.

The normalizing average retrieval order was calculated when data sizesof the picture feature amount 1 of the orthogonal conversion coefficientand the picture feature amount 4 were made substantially equal to eachother and data sizes of the picture feature amount 2 of the colorhistogram information and the picture feature amount 3 were madesubstantially equal to each other, and then the respective picturefeature amounts were used to carry out the retrieval.

FIG. 7 is a view showing the normalizing average retrieval order whenthe orthogonal conversion coefficient is used as the picture featureamount, and FIG. 8 is a view showing the normalizing average retrievalorder when the color histogram information is used as the picturefeature amount.

From FIGS. 7, 8, as compared with the case when the orthogonalconversion coefficient and the color histogram information are generatedfrom the raw picture, the normalizing average retrieval order isincreased only by 0.03 at most in the case of the usage of the colorhistogram information and the orthogonal conversion coefficientgenerated by the conversion of the picture feature amount. Thus, it isunderstood that there is scarcely any fear of the reduction in theretrieval performance.

For this reason, both the color histogram information and the orthogonalconversion coefficient can be treated by he same similarity calculator.Thus, it is understood that the picture retrieving apparatus having thesimplified system configuration can be provided without any large dropin the retrieval performance.

Moreover, it is possible to provide the picture retrieving apparatusthat can carry out the retrieval, whether the kind of the picturefeature amount is the orthogonal conversion coefficient or the colorhistogram information.

As mentioned above, the embodiments of the present invention have beendescribed with reference to the drawings. However, the actualconfiguration is not limited to the embodiments. Even if, the change inthe design without departing from the scope and spirit of the presentinvention is made, it is included in the present invention. For example,the kind of the color in the color histogram information is not limitedto the particular color space. For example, any color space can be usedamong an RGB color space, a YCbCr color space, an HSV color space andthe like. Also, in calculating the average color, it is not necessary toadd all the component colors. Only a part of the colors may be used.

Also, if the method according to the present invention is attained byusing a software, it can be easily attained by installing a program forcarrying out the method of the present invention in a processingapparatus including: a main control unit containing a central processingunit (CPU); a memory unit composed of ROM, RAM, HOD and the like; aninput output unit having a man-machine interface; a display; and thelike, as a hardware configuration. Such a program is prepared in advancein a condition that it is recorded in a medium of any type that can beread by the processing apparatus. Then, if the similar system isincluded, the same method can be universally attained by reading such aprogram. As such a medium, it is possible to use any type of a magnetictape, a floppy disc, a CD-ROM and the like.

As mentioned above, according to the picture retrieving apparatus andthe picture retrieving method of the present invention, the picturesimilar to the query picture can be specified from the picture targetedfor the retrieval, on the basis of the similarity of the picture featureamount, irrespectively of the kind of the used picture feature amount,by converting the picture feature amount and then unifying the kind. Atthis time, the retrieval performance is not largely dropped. Also, thesame similarity calculator can be applied to the different picturefeature amount. Thus, it is possible to simplify the configuration ofthe picture retrieving apparatus.

INDUSTRIAL APPLICABILITY

Typically, the picture retrieving method using the feature of thepicture has been traditionally used to retrieve the similar picture fromthe accumulated pictures. In the present invention, the comparativeretrieval can be done between the different kinds of the picture featureamounts. For example, if the picture feature amount of the query pictureis the orthogonal conversion coefficient, the retrieval can be performedon the database having the color histogram information of the retrievaltarget picture as the feature amount.

1. A picture retrieving apparatus comprising: a first color histogramgenerating section which outputs a color histogram data of each of aplurality of pictures; a second color histogram generating section whichoutputs a color histogram data of a query picture; and a color histogramsimilarity calculator which calculates a similarity between said colorhistogram data of said query picture and said color histogram data ofone of said plurality of pictures as a retrieval target picture, whereinsaid first color histogram generating section comprises: an orthogonaltransform coefficient memory which stores a set of orthogonal transformcoefficients of each of said plurality of pictures; and a first picturefeature data converter which converts said set of orthogonal transformcoefficients of said retrieval target picture read from said orthogonaltransform coefficient memory into said color histogram data of saidretrieval target picture, and outputs said color histogram data of saidretrieval target picture to said color histogram similarity calculator.2. The picture retrieving apparatus according to claim 1, wherein saidfirst picture feature data converter comprises: an inverse orthogonaltransformer which decodes said set of orthogonal transform coefficientsof said retrieval target picture to convert into said retrieval targetpicture; and a color histogram data generator which generates said colorhistogram data of said retrieval target picture and outputs to saidcolor histogram similarity calculator.
 3. A picture retrieving apparatuscomprising: a first color histogram generating section which outputs acolor histogram data of each of a plurality of pictures; a second colorhistogram generating section which outputs a color histogram data of aquery picture; and a color histogram similarity calculator whichcalculates a similarity between said color histogram data of said querypicture and said color histogram data of one of said plurality ofpictures as a retrieval target picture, wherein said second colorhistogram generating section comprises: a second picture feature dataconverter which converts a set of orthogonal transform coefficients ofsaid query picture into said color histogram data and outputs to saidcolor histogram similarity calculator.
 4. The picture retrievingapparatus according to claim 3, wherein said second picture feature dataconverter includes: an inverse orthogonal transformer which decodes saidset of orthogonal transform coefficients of said query picture toconvert into said query picture; and a color histogram data generatorwhich generates said color histogram data of said query picture andoutputs to said color histogram similarity calculator.
 5. A pictureretrieving method comprising the steps of: (a) outputting a colorhistogram data of each of a plurality of pictures; (b) outputting acolor histogram data of a query picture; and (c) calculating asimilarity between said color histogram data of said query picture andsaid color histogram data of one of said plurality of pictures as aretrieval target picture, wherein said step of (a) outputting comprisesthe steps of: providing a set of orthogonal transform coefficients ofeach of said plurality of pictures; (d) converting said set oforthogonal transform coefficients of said retrieval target picture intosaid color histogram data of said retrieval target picture; andoutputting said color histogram data of said retrieval target picture.6. The picture retrieving method according to claim 5, wherein said stepof (d) converting comprises the steps of: decoding said set oforthogonal transform coefficients of said retrieval target picture toconvert into said retrieval target picture; and generating said colorhistogram data of said retrieval target picture.
 7. A picture retrievingmethod comprising the steps of: (a) outputting a color histogram data ofeach of a plurality of pictures; (b) outputting a color histogram dataof a query picture; and (c) calculating a similarity between said colorhistogram data of said query picture and said color histogram data ofone of said plurality of pictures as a retrieval target picture, whereinsaid step of (b) outputting comprises the step of: (e) converting a setof orthogonal transform coefficients of said query picture into saidcolor histogram data.
 8. The picture retrieving method according toclaim 7, wherein said step of (e) converting comprises the steps of:decoding said set of orthogonal transform coefficients of said querypicture to convert into said query picture; and generating said colorhistogram data of said query picture.